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We share results from a quasi-experimental study in which we compared achievement between 
traditional lecture-based and computer-based sections of college algebra on a common multiple 
choice exam as well as performance on problem solving items. Students in the computer-based 
group performed better on the final exam and were also more likely to complete the problem 
solving tasks correctly than students in the traditional courses. However, the computer-based 
group showed limited ability to interpret an equation and relate it to a contextual situation. 
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The use of “emporium” style delivery formats for basic college mathematics courses is 
spreading in the United States (Twigg, 2011). These courses are held in large computer labs, 
where the majority of instruction is provided through interaction with a computer program. The 
software provides examples, explanations, videos, opportunities to practice, and varying 
degrees of feedback on incorrect solutions. Emporium courses also employ trained instructional 
assistants who provide one-on-one “just in time” help (Hodges & Brill, 2007). The limited 
number of empirical studies conducted in these sites indicate positive effects on attendance, 
pass rates, performance on end-of-course exams, and future enrollment in math courses 
(Taylor, 2008; Twigg, 2011). Despite theses findings, it is unclear what mathematical 
knowledge students gain from learning in computer-based environments. 


Objectives 

The purpose of this paper is to report quantitative findings from a comparative study 
regarding the impact of computer-based versus face-to-face instruction for college algebra and 
to analyze students’ solution strategies on non-routine mathematics tasks. This research is an 
important first step towards determining the extent to which these two contrasting learning 
environments influence mathematical knowledge and the degree to which students in these 
different environments are able to solve non-routine tasks in mathematics. 

According to Twigg (2011), the success of the mathematics emporium model can be 
attributed to several characteristics, including: (a) students do problems rather than watching a 
lecturer do the problems, (b) students spend more time on what they do not understand and less 
time on what they already know, (c) students get timely assistance from the software as well as 
trained instructional assistants, and (d) students must master a requisite skill before they proceed 
to a more advanced skill. The characteristics listed above correspond to the broad finding that 
developing procedural skill requires teaching that incorporates modeling the skill followed by a 
significant amount of error-free practice on the part of the learner (Hiebert & Grouws, 2007). 

At some universities, blended instruction incorporates some of these characteristics without 
removing the face-to-face lectures, still producing significant learning gains, and in some cases, 
improved self-efficacy beliefs for students (Hagerty & Smith, 2005; Hagerty, Smith, & Goodwin 
2010; Kendricks, 2011; Taylor, 2008). Kodippilli and Senaratne (2008), found that passing rates 
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were 43% higher in a class that used MyMathLab, a software commonly used in mathematics 
emporia, than in a class where it was not used. 

Despite this reported success, the emporium model raises concerns about the potential lack of 
attention to conceptual understanding. Erlwanger’s (1973) classic description of the elementary 
student “Benny” serves as a reminder and warning about the kinds of misconceptions that can 
result from an individualized program that emphasizes correct answers and procedures without 
attending to students’ understanding of mathematical concepts. This study attends to this concern 
by providing students an opportunity to apply their mathematical knowledge to problems that 
may seem non-routine and to explain how they think about the mathematical ideas and skills 
they use to solve the problems. 


Methods 
Context and Sample 

In the Spring of 2012 the mathematics department at a mid-sized suburban university offered 
twelve sections of Intermediate Algebra. Six sections were held in a computer-based (CB), 
emporium-style environment and six sections were taught in traditional face-to-face (F2F) 
classes by adjunct professors who typically teach the course. The students did not know at the 
time of enrollment whether they would be assigned to the CB or F2F learning environment. 

A quasi-experimental matched group design was used to explore the impact of the two 
settings on student learning, followed by finer grained analyses. As such, the sample includes 
three levels of participants, determined by data access granted by participants. Level | includes 
all students enrolled in Intermediate Algebra during the Spring of 2012 with an exam score 
(ncp=134, npor=192), Level 2 is made up of a subset of students from each learning environment 
(ncp=73, Np2F=50) and includes additional student-level predictors, and Level 3 participants are a 
subset of the Level 2 participants with completed non-routine tasks (ncp=38, Nf2F=24). 

Data Sources and Analysis 

Each student took a common multiple-choice final exam at the conclusion of the semester 
through their normal means: CB via their computer program and F2F in their classroom. For the 
Level 1 analysis, a t-test was used to determine if there was a difference in the group means 
between the CB and F2F sections. To control for initial differences in the two student samples, as 
well as to explore the factors that influenced student achievement on the final exam between the 
two treatments, the Level 2 analysis included ANCOVA’s with student-level variables for SAT 
math (prior achievement measure), high school GPA and Algebra II grade, and university GPA. 

To determine if there were differences between CB and F2F in how students solved the 
problem-solving tasks, the Level 3 data include student responses on four problem-solving items. 
These items were given near the end of the semester for a small amount of extra-credit. They 
were completed individually, in class, on paper, and were based on major themes of the 
Intermediate Algebra course: systems of equations and quadratic functions. They were designed 
to represent various degrees of transfer from the kinds of procedural tasks they solved on the 
final exam. In this paper, we report on two problems, shown below: 

1. Cindy bought 3 burgers and 2 soft drinks for $10.50. At the same restaurant, Sean 
bought 4 burgers and 3 soft drinks for $14.50. How much does | burger cost? 
2. Fora concert, the income from ticket sales is estimated to depend on ticket price 


according to the following equation: t(x) = 25x? + 750x, where x is price per ticket. 
a. Will increasing the price of the ticket always result in greater income? Explain 
how you know. 
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b. Find the ticket price where the concert will make the maximum amount of 
money, according to the equation. (Hirsch, Fey, Hart, Schoen, & Watkins, 
2008) 

Two members of the research team coded the students’ solution strategies and whether the 
students answered the problem correctly. The two coders reached a 96% agreement on correct 
versus incorrect and 87% on the strategies the students used. 

Level 1: All Students 

On average, student achievement on the final exam was higher for students in the CB 
(u=70.75, o =15.29) than for students in the F2F sections (u =65.49, o =13.12). This difference 
was significant (t=-3.33, p" .001), which represented a small effect size (r=.182) (Cohen, 1988). 
Level 2: Subset of Students with Additional Predictors 

For the students participating in the Level 2 study, there was no significant difference 
between students enrolled in the F2F section (u =68.59, o =12.80) and those in the CB sections 
(u =70.60, o =16.01) (t=-0.74, p=.461). The ANCOVA, using SAT math as a covariate, showed 
a larger effect size, SAT math score significantly predicted students’ final exam score (F=43.93, 
p<.0001). More interesting, when controlling for SAT math, the effect of the mode of delivery 
favored CB (F=3.58, p=.06). Furthermore, when the mode of delivery, SAT math, and the 
interaction between mode of delivery and SAT math were included as predictors, Mode was 
significantly related to final exam score (F=4.32, p=.040). In addition, Mode*SAT math was a 
significant predictor of student achievement (F= -2.31, p=.029). These findings suggest that there 
is a significant relationship between SAT math, the mode of delivery, and the final exam, such 
that, on average, students with higher SAT scores tended to perform better in the CB setting. 
Conversely, students with lower SAT scores tended to perform better in the F2F setting. 

Level 3: Problem-Solving Tasks 

On Task 1, 76% of the CB students answered the problem correctly, whereas only 58% of the 
F2F students responded with a correct answer. Students used strategies such as guess and check, 
elimination, or other algebraic strategies. Interestingly, students used different strategies 
depending on their learning environment (Table 1). Students in the F2F classes were more likely 
to guess and check and less likely to use an elimination strategy than those in the CB classes. 
These results show that although the CB students were more successful in solving the task, there 
was less variance in their strategies, whereas F2F students used a greater variety of strategies. 


Table 1: Task 1 F2F and CB Percentage of Students’ Solution Strategy 


Guess Elimination Algebraic Other 
Face-to-Face 46% 21% 16% 16% 
Computer Based 27% 60% 12% 0% 


The first part of Task 2 asked if increasing the ticket prices would always result in increased 
income. Of the F2F students, 78% correctly indicated that income would not continue to 
increase, justifying this conclusion by appealing to the problem context, the negative coefficient 
in the equation, or by testing values. In comparison, only 58% of the CB students correctly 
indicated that the income would not continue to increase. All of the F2F students gave a 
justification for their conclusion, but 18% of the CB students did not give any justification. For 
the second part, where students had to find the ticket price that would maximize income, only 
31% of the F2F students found the correct ticket price, whereas 63% of the CB students found 
that a ticket price of $15 would maximize income. Students used many different strategies to 
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answer the second part of the problem (Table 2). All 37% of the CB students using the vertex 
equation obtained the correct answer, and, of the 32% of CB students who solved the problem by 
testing different values for the ticket price, all but two of them (83%) got the right answer. In 
contrast, none of the F2F students used the vertex equation and less than half of them found the 
correct answer by testing ticket prices. This problem reveals that while the CB students were 
more likely to use an algebraic procedure and to solve the problem correctly, they were less 
likely to recognize that the income would not continue to rise as the ticket price increased. 


Table 2: Task 2 F2F and CB Percentage of Students’ Solution Strategy 


Ver Tests} Complete Sq./ Blank or Attempts 
tex Value} Factor/Graph No to 
Face-to-Face 0 61% 8 Z 4 
Computer 2 32% 1 2 0 
Significance 


As universities across the nation transition to more cost effective means of delivering 
mathematics instruction, it is important to verify that these methods of delivery are effective, not 
only in promoting students’ procedural fluency but also in providing students with a rich 
experience that shows them the connections behind algorithmic procedures. This study suggests 
that while students in emporium-style settings may gain procedural fluency, they may be limited 
in their ability to interpret mathematical objects and relate them to contextual situations. Given 
the limited number of empirical studies on the impact of emporium-style math courses, more 
work is needed to explore not only systemic effects but also the nature of the mathematical 
knowledge that students gain in these settings. 
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